Identifying Social Deliberative Behavior from Online Communication - A Cross-Domain Study

In this paper we describe automatic systems for identifying whether participants demonstrate social deliberative behavior within their online conversations. We test 3 corpora containing 2617 annotated segments. With machine learning models using linguistic features, we identify social deliberative behavior with up to 68.09% in-domain accuracy (compared to 50% baseline), 62.17% in-domain precision, and 84% in-domain recall. In cross-domain identification tasks, we achieve up to 55.56% cross-domain accuracy, 59.84% cross-domain precision, and 86.58% cross-domain recall. We also discover linguistic characteristics of social deliberative behavior. In the context of identifying social deliberative behavior, we offer insights into why certain machine learning models generalize well across domains and why certain domains pose great challenges to machine learning models.